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INTRODUCTION 


The  Military  Interoperable  Digital  Hospital  Testbed  (MIDHT)  is  a  five-year  program  of  research 
to  develop  a  real-world  testbed  environment  in  Southwestern  Pennsylvania.  The  purpose  is  to 
research  and  evaluate  Health  Information  Exchange  (HIE)  and  health  information  technology 
(HIT)  and  services  that  make  health  information  readily  available  to  consumers  and  providers. 
Ideally  this  will  allow  for  the  secure  transfer  of  information  between  private  sector  rural 
providers,  federal  partners  and  patients.  MIDHT  has  defined  requirements  and  solutions  to 
optimize  healthcare  resources  for  rural  communities  and  identified  lessons  learned  and  best 
practices  that  benefit  both  the  global  Military  Health  System  (MHS)  environment  and 
stakeholders  in  the  region.  The  Department  of  Defense  (DoD)  and  Conemaugh  Memorial 
Medical  Center  (CMMC)  have  common  requirements  for  HIE,  connecting  disparate  systems  and 
providers  and  enabling  secure  provider-provider  and  provider-consumer  e-communications. 
Minimal  evidence  is  available  on  what  business,  clinical  and  technical  solutions  can  be  used  to 
overcome  the  lack  of  specialists,  infrastructure  and  geographical  barriers  associated  with  the 
delivery  of  care  in  rural  communities. 


BODY 


Arm  1:  Longitudinal  Study  for  Use  of  Interoperable  Accessible  Health 
Information  Exchange  Services  and  Technologies  in  Rural 
Communities  (A  -  15835.2,  A  -  16192.1). 

This  arm  focuses  on  ways  a  rural  environment  can  capitalize  on  the  use  of  health  information 
network  (HIN)  services  and  technologies  to  promote  interoperability  between  disparate  entities 
such  as  TNPs,  private  sector  health  systems,  and  DoD  facilities.  MIDHT  has  investigated 
attitudes,  usability,  and  effectiveness  of  HIN  services  by  rural  providers,  including  the  effect  of 
the  use  of  HIE  tools  by  provider  groups,  TRICARE  providers,  and  three  CHS  facilities  on  their 
business  practices  and  process  flows.  Research  initiatives  have  focused  on  the  impact  of  an 
electronic  health  record  implementation  using  instruments  utilized  in  year  1 .  Additionally, 
research  initiatives  evaluated  the  ability  to  electronically  access  digital  radiology  images  and  how 
this  system-wide  functionality  affected  the  delivery  of  patient  care  within  a  rural  health  care 
system,  to  include  an  analysis  on  provider  productivity,  throughput,  duplicative  testing  and 
continuity  of  care.  Finally,  an  assessment  of  the  volume  of  cases  that  Conemaugh  physicians 
have  with  SSA  regarding  veteran/military  disability  claims  and  provider  satisfaction  with  the 
existing  SSA  process  have  been  completed. 
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Subtask  1.1  Assess  changes  in  provider  workflows  and  efficiency  resulting  from  the 
implementation  of  an  ambulatory  electronic  medical  record. 


Descriptive  Analysis  of  EHR  Survey  Results  (n=26): 

Overall,  the  staff  at  Portage  and  NORCAM  indicated  an  ambivalence  regarding  satisfaction  of 
the  EHR  implementation  with  almost  one  third  stating  a  neutral  satisfaction.  When  looking  at 
position  type,  no  apparent  trends  are  noted  (Note:  “Other”  most  likely  includes  some  physicians). 
These  satisfaction  results  are  much  lower  than  those  reported  by  DesRoches  et  al  regarding  a 
2008  national  survey  of  physicians  on  EHR’s.  Only  34%  of  CMMC  staff  is  satisfied  whereas 
90%  of  physicians  surveyed  nationally  were  satisfied.  Despite  the  finding  of  neutral  satisfaction, 
descriptive  analysis  of  certain  questions  indicates  a  positive  perception  of  the  EHR.  Aggregating 
the  answer  choices  for  question  twenty  (q0020)  as  either  difficult  or  easy  shows  that  a  majority  of 
respondents  indicated  the  following  activities  are  easier  when  using  the  EHR  as  compared  to 
previous  routines: 

•  Documenting  allergies 

•  Documenting  CPT  and  ICD-9  codes  for  billing  purposes 

•  Keeping  problem  lists  updated 

•  Reviewing  laboratory  and  radiology  results 

•  Writing  and  renewing  prescriptions 

•  Monitoring  medication  safety  during  prescribing 

•  Communicating  referral  information  to  specialists 

Furthermore,  approximately  50%  of  respondents  agree  that  the  EHR  has  enabled  them  to 
accomplish  tasks  more  quickly,  has  enhanced  job  effectiveness  and  made  it  easier  to  do  their  job. 
Whereas  45%  of  respondents  state  they  work  longer  hours  to  see  the  same  number  of  patients. 
Also,  no  consensus  that  the  EHR  has  caused  disruptions  to  their  workflow  was  found. 

Written  comments  most  often  cited  suggest  that  staff  appreciate  not  having  to  pull  and  refile 
paper  charts,  access  to  hospital  information  (e.g.  lab/rad  results)  is  much  easier  (direct  interface 
to  Allscripts),  and  benefits  of  ePrescribing.  On  the  negative,  staff  members  are  frustrated  with 
the  multiple  screens  and  not  being  able  to  quickly  find  information,  creating  clinical  notes  is  too 
cumbersome,  determining  which  tests  were  ordered  was  more  difficult  and  more  training  is 
needed. 

Although  the  significance  of  the  following  statistical  analysis  cannot  be  interpreted  as  definitive 
due  violations  resulting  from  the  sample  size,  the  results  due  generally  support  the  conclusions 
drawn  from  the  descriptive  analysis  above. 

Statistical  Analysis  of  EHR  Survey  Results  (n=26): 


Crosstabs  were  performed  over  questions  1-19  (rows)  on  questions  22,  27,  28,  29,  30  and  31 
(columns).  Chi-square  was  used  to  test  for  significance  between  the  columns  over  the  rows. 
Unfortunately,  the  sample  size  (N=26)  produced  untenable  cell  counts  and  significance  testing 
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could  not  be  performed.  Furthermore,  a  crosstab  of  question  30  and  31  did  not  produce  a 
significant  result  using  Chi-square;  nor  did  question  22  with  30.  Analyzed  independently, 
questions  20  and  21  yielded  slightly  positive  responses,  overall.  The  low  sample  size  overall  and 
decline  in  responses  over  time  prevented  a  longitudinal  analysis  of  survey  data  as  designed. 
Please  refer  to  Appendix  1  for  a  copy  of  the  survey  results. 


S02_q0001 

S02_q0002 

S02_q0003 

S02_q0004 

S02_q0005 

S02_q0006 

S02_q0007 

S02_q0008 

S02_q0009 

S02_q001 0 

S02_q001 1 

S02_q001 2 

S03_1_q0013 

S03_2_q0014 

S03_3_q0015 

S03_4_q0016 

S03_5_q0017 

S03_6_q0018 

S03_7_q0019 


S04_3_q0022 

0.274 
0.063 
0.020 
0.018 
0.011 
0.225  (0.378) 
0.255 
0.320 
0.819 
0.206 
0.161 
0.151 
0.030 
0.070 
0.362 
0.332 
0.041 
0.023 
0.229 


S04_8_q0027 

0.108 
0.568 
0.644 
0.433 
0.719 
0.431  (0.695) 
0.988 
0.486 
0.793 
0.930 
0.550 
0.253 
0.136 
0.574 
0.113 
0.756 
0.105 
0.134 
0.596 


S04_9_q0028 

0.295 

0.615 

0.390 

0.356 

0.347 

0.018 

0.398 

0.378 

0.727 

0.059 

0.223 

0.210 

0.511 

0.338 

0.957 

0.966 

0.556 

0.022 

0.055 


S041 0_q0029 

0.008 

0.032 

0.188 

0.728 

0.739 

0.743 

0.379 

0.389 

0.384 

0.837 

0.753 

0.012 

0.185 

0.255 

0.424 

0.238 

0.018 

0.255 

0.753 


S04_11_q0030 

0.200 
0.511 
0.267 
0.080 
0.817 
0.027  (0.073) 
0.203 
0.037 
0.039 
0.022 
0.759 
0.648 
0.626 
0.763 
0.917 
0.799 
0.317 
0.378 
0.039 


S04_12_q0031 

0.085 

0.005 

0.037 

0.102 

0.012 

0.053 

0.013 

0.094 

0.322 

0.049 

0.567 

0.330 

0.169 

0.592 

0.920 

0.691 

0.093 

0.160 

0.133 


NOTES: 


1)  Values  in  this  table  represent  the  Pearson  Chi-square  result  except  where  indicated  below. 

2)  ( )  indicate  the  asymptotic  significance  as  calculated  by  the  Fisher's  Exact  Test  for  a  2x2  matrix 

3)  italics  indicate  that  fewer  than  25%  of  the  cells  have  an  expected  count  <  5 

4)  All  results  (except  those  in  italics)  result  from  a  matrix  that  has  at  least  33%  of  the  cells  with  an 
expected  count  <  5.  As  such,  these  results  should  not  be  used  to  make  concrete  inferences  as  to 
significance. 


Table  1.  Chi-Square  p-values  for  Crosstabs  of  Survey  Questions 
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Crosstab 


No 

Yes 

Total 

Count 

5a 

13a 

18 

No 

S04  11 

%  within 

S04  3  q0022 

71.4% 

72.2% 

72.0% 

_q0030 

Count 

2a 

5a 

7 

Yes 

%  within 
S04_3_q0022 

28.6% 

27.8% 

28.0% 

Count 

7 

18 

25 

Total 

%  within 
S04_3_q0022 

100.0% 

100.0% 

100.0% 

Each  subscript  letter  denotes  a  subset  of  S04_3_q0022  categories 
whose  column  proportions  do  not  differ  significantly  from  each  other 
at  the  .05  level. 


Chi-Square 


Tests 


Asymp. 

Exact 

Exact 

Sig. 

Sig. 

Sig. 

Value 

df 

(2-sided) 

(2-sided) 

(1 -sided) 

Pearson  Chi- 
Square 

.002a 

1 

.968 

Continuity 

Correction13 

.000 

1 

1.000 

Fisher's  Exact 
Test 

1.000 

.663 

N  of  Valid 

25 

Cases 

a.  1  cell  (25.0%)  1 

has  an  expected  count  less  than  5.  The 

minimum  expected  count  is  1 .96. 
b.  Computed  only  for  a  2x2  table 

Table  2.  Crosstab  Results  and  Chi-Square  p- value  for  Survey  Questions  11  and  22 


Crosstab 


|  S04_3_q0022  | 

Total 

No 

Yes 

Count 

Dissatisfied  %  within 

S04_3_q0022 

4a 

57.1% 

2b 

11.1% 

6 

24.0% 

VERY  C0Unt 
Dissatisfied  °/o  within 

S04  3  q0022 

la 

14.3% 

2a 

11.1% 

3 

12.0% 

S04  12  f  .  Count 

q0031  neutral  %  within 

S04  3  q0022 

2a 

28.6% 

5a 

27.8% 

7 

28.0% 

.  .  Count 

somewhat 

satisfied  0//°  w'thin 

S04_3_q0022 

0a 

.0% 

8b 

44.4% 

8 

32.0% 

VERY  Count 
satisfied  0/°  w'thin 

S04_3_q0022 

0a 

.0% 

la 

5.6% 

1 

4.0% 

Count 

Total  %  within 

S04_3_q0022 

7 

100.0% 

18 

100.0% 

25 

100.0% 

Each  subscript  letter  denotes  a  subset  of  S04_3_q0022  categories  whose 
column  proportions  do  not  differ  significantly  from  each  other  at  the  .05 

level. 


Chi-Square  Tests 


Value 

df 

Asymp.  Sig. 
(2-sided) 

Pearson 

Chi- 

Square 

7.993a 

4 

.092 

N  of 
Valid 
Cases 

25 

a.  8  cells  (80.0%)  have  expected 
count  less  than  5.  The  minimum 
expected  count  is  .28. 


Table  3.  Crosstab  Results  and  Chi-Square  p- value  for  Survey  Questions  12  and  22 
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S04_11_q0030  *  S04_12_q0031  Crosstabulation 


S04_12_q0031 

Total 

Dissatisfied 

VERY 

Dissatisfied 

neutral 

somewhat 

satisfied 

VERY 

satisfied 

S04_11_q0030  No  Count 

%  within  S04_12_q0031 

2a 

33.3% 

2a 

66.7% 

7a 

87.5% 

7a 

87.5% 

la 

100.0% 

19 

73.1% 

Yes  Count 

%  within  S04_12_q0031 

4a 

66.7% 

la 

33.3% 

la 

12.5% 

la 

12.5% 

0a 

.0% 

7 

26.9% 

Total  Count 

%  within  S04_12_q0031 

6 

100.0% 

3 

100.0% 

8 

100.0% 

8 

100.0% 

1 

100.0% 

26 

100.0% 

Each  subscript  letter  denotes  a  subset  of  S04_12_q0031  categories  whose  column  proportions  do  not  differ  significantly  from  each 
other  at  the  .05  level. 


Chi-Square  Tests 


Value 

df 

Asymp.  Sig. 
(2-sided) 

Pearson  Chi-Square 

N  of  Valid  Cases 

6.940a 

26 

4 

.139 

a.  8  cells  (80.0%)  have  expected  count  less  than  5. 
The  minimum  expected  count  is  .27. 


Table  4.  Crosstab  Results  and  Chi-Square  p-value  for  Survey  Questions  30  and  31 


Statistical  Analysis  of  Productivity  Data  (n  =  6  providers  (control),  5  providers  (EHR)): 

Initial  statistical  analysis  obviated  the  necessity  for  additional  data  points  to  achieve  sufficient 
symmetry  and  homogeneity  to  produce  an  analyzable  dataset.  A  protocol  modification  was 
necessary  to  further  clarify  the  time  period  to  be  used  (January  2007  -  February  2010)  for 
analysis,  which  was  approved  by  Conemaugh  IRB  on  December  9,  201 1. 

Quarterly  data  was  obtained  for  years  2007  through  2011.  Both  a  MANOVA  and  RM- 
(m)ANOVA  were  attempted;  however,  underlying  assumptions  for  those  tests  were  violated.  The 
data  was  reorganized  and  recast  to  conform  to  a  paired-t  and  a  standard  independent  t-test.  Since 
Year  2007  only  contained  data  for  the  following  variables,  Charges,  Units,  Encounters,  and 
Office  Hours  Worked,  it  could  not  be  included  in  the  paired-t  or  independent  t-test  derived  from 
the  paired-t  setup.  Furthermore,  the  calendar  quarters  for  all  years  were  not  consistent  due  to  the 
implementation.  To  minimize  the  impact  of  potential  periodicity,  the  data  pairing  for  the  paired-t 
used  2009,  2010,  and  201 1  data.  The  paired-t  test  was  used  to  investigate  the  within-group 
change  between  the  post  and  pre  time  periods.  Due  to  the  testing  of  multiple  dependent  variables, 
the  test-wise  alpha  was  adjusted  so  that  the  overall  alpha  remained  0.05.  The  following 
calculated  variables  were  produced  so  as  to  provide  for  a  more  direct  comparison  between  the 
control  and  intervention  groups:  Charge/Unit,  Encounters/Hr,  and  Total  RVUs  per 
Encounters/Hour.  The  difference  (post-pre)  was  calculated  and  a  standard  t-test  was  applied  to 
test  if  the  mean  difference  was  statistically  significant  between  groups.  No  statistical 
significance  was  found. 

Despite  the  finding  of  no  statistical  significance  between  the  control  and  intervention  group,  the 
change  (post  -  pre)  in  the  mean  difference  of  encounters  per  hour  improved  for  the  intervention 


Page  8  of  44 


group  and  not  the  control  group  (0.28  vs  -0.086).  A  similar  improvement  was  also  noted  for 
Total  RYUs  (546  vs  316).  In  other  words,  EHR  users  likely  realized  an  increase  in  patients  per 
hour  and  productivity  when  compared  to  the  paper-based  physician  offices. 

Study  Closure: 

Subject  study  was  closed  with  Conemaugh  IRB  on  April  2,  2012.  Final  report  and  supporting 
documents  were  delivered  to  USAMRMC  ORP  HRPO  and  found  to  be  acceptable  on  April  19, 
2012  (refer  to  Appendix  B). 

Subtask  1.2  Enhance  the  service-based  HIE  infrastructure  and  services  to  support  further 
exchange  of  digital  medical  imaging  information  in  a  rural  setting. 

****  Task  completed  during  previous  periods**** 

Subtask  1.3  Research  and  evaluate  the  ability  to  electronically  exchange  digital  images  and 
how  this  functionality  will  affect  the  delivery  of  patient  care  within  a  rural  health 
care  system,  to  include  an  analysis  on  provider  productivity,  throughput, 
duplicative  testing  and  continuity  of  care. 


Study  Closure: 

This  study  has  been  completed  and  closed  with  Conemaugh  IRB  on  July  14,  2011.  Final  report 
and  supporting  documents  were  delivered  to  USAMRMC  ORP  HRPO  and  found  to  be 
acceptable  on  August  29,  2011  (refer  to  Appendix  C). 

Manuscript: 

A  study  manuscript  was  completed  and  submitted  to  Radiology  on  April  12,  2012. 

Unfortunately,  CMMC  received  notification  on  May  10,  2012  that  it  was  not  accepted  for 
publication.  Reviewer  comments  have  been  discussed  by  the  study  team  and  minor  edits  are 
being  completed  in  order  to  submit  to  Telemedicine  and  eHealth  in  July  2012.  Please  refer  to 
Appendix  D  for  a  copy  of  the  manuscript. 

Subtask  1.4  Deploy  (via  portal  technology)  a  pilot  demonstration  of  the  electronic  exchange  of 
private  sector  ambulatory  medical  records  with  the  DoD  and  other  selected 
stakeholders  using  test  data. 

****  Task  completed  during  previous  periods**** 
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Subtask  1.5 


Subtask  1.6 


Subtask  1.7 


Perform  a  technical  feasibility  study  to  focus  on  repurposing  the  BHIE-AHLTA 
web  services  toward  the  existing  NHIN  Federal  Adapter  for  the  purpose  of 
standards  based  exchange  of  Military  Health  System  data  domains  with  private 
sector  partners. 

****  Xask  completed  during  previous  periods**** 


Begin  development  on  a  private  sector  version  of  the  Federal  Gateway/ Adapter 
(work  to  be  based  on  the  code  that  is  anticipated  to  be  available  from  ONC)  using 
interoperable  HITSP  standards  to  progress  the  goals  of  this  national  effort. 

****  Task  completed  during  previous  periods**** 

Perform  an  assessment  of  the  volume  of  cases  that  Conemaugh  physicians  have 
with  SSA  regarding  veteran/military  disability  claims  and  assess  provider 
satisfaction  with  existing  SSA  process  for  information  gathering  and  submission. 

****  Task  completed  during  previous  periods**** 
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Arm  2:  The  Impact  of  Consumer  Informatics  in  the  Chronic  Care 
Model:  Metabolic  Syndrome  and  Gestational  Diabetes  in  a 
Rural  Setting  (A-15835.1). 

This  arm  focuses  on  finding  innovative  solutions  to  slow  down  the  growing  epidemic  of 
metabolic  syndrome  in  the  United  States  using  consumer  informatics.  A  personal  health  record 
(PHR)  is  offered  to  enrolled  randomized  subjects  for  six  months.  Subjects  are  able  to  create  a 
PHR  and  communicate  electronically  with  their  physician  and  staff  members.  A  total  of  five 
physician  practices  have  participated  in  the  study  with  the  recent  addition  of  a  gestational 
diabetes  group.  Changes  in  clinical  outcome  measures  are  compared  to  a  control  group, 
consisting  of  study  specific  enrolled,  randomized  subjects.  In  addition,  qualitative  data  is  being 
assessed  through  a  survey  instrument  and  PHR  usage  data  is  collected  to  make  sound  study 
conclusions. 

Subtask  2.1  Deploy  HIE  tools  for  patient  and  community  outreach  in  varied  rural 
environments. 

Subject  recruitment  for  the  respective 
study  has  been  challenging.  Interest 
has  been  greatest  from  patients 
deriving  from  suburban  physician 
offices  whereas  anecdotal  concerns 
persist  regarding  Internet  access  and  an 
elderly  population  in  rural 
communities.  Patients  and  providers, 
from  our  multi-year  experience,  are 
clearly  not  interested  and  ready  for 
personal  health  records  as  a  tool  in 
managing  healthcare  matters.  Our 
experience  directly  aligns  with  national 
adoption  rates  and  attitudes  towards  PHR 

The  opportunity  to  create  secure,  online  personal  health  records  and  send  electronic  messages  to 
providers  through  RelayHealth  has  not  been  well  received  by  patients.  Recruitment  challenges 
align  with  low  adoption  of  said  technology  nationwide.  The  California  Healthcare  Foundation 
reports  that  only  7%  of  U.S.  adults  use  some  sort  of  PHR,  with  51%  of  access  offered  by  their 
health  insurance  plan  (1).  In  addition,  consumers  in  rural  areas  have  known  issues  with  access 
to  the  Internet,  low  self-awareness  of  their  health  conditions  and  an  elderly  population  that  is 
hesitant  with  technology  adoption.  Finally,  consumers  have  stated  concerns  with  putting  their 
health  information  online  due  to  theft  and  privacy  invasion. 


MIDHT  PHR  Enrollment  Summary 

Description 

Reporting 

Period 

Total 
to  Date 

Inquiries 

84 

195 

Screenings 

41 

79 

Screen  Failures 

4 

9 

Enrolled  (Metabolic  Syndrome) 

33 

64 

Enrolled  (Gestational  Diabetes) 

5 

5 

Withdrawals 

5 

11 

Active 

25 

Completed 

14 

33 

’s. 

Table  5.  Summary  Enrollment. 
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In  order  to  assess  patient  interest  in  the  study,  CMMC  staff  has  used  various  recruiting  strategies 
during  the  past  year,  including: 

■  Direct  patient  letters 

■  Advertisements  in  numerous  newspapers 

■  Facebook  postings 

■  Flyers  and  posterboards  in  participating  physician  offices 

■  Phone  On  Hold  messages 

■  Global  emails 

■  Face-to-face  meetings  with  physicians 

■  Health  Fairs 

■  Metabolic  Syndrome  lectures 

Enrollment  has  closed  for  the  metabolic  syndrome  arm  to  allow  time  for  subjects  to  complete  the 
intervention,  data  analysis  and  write-up  of  results  before  contract  termination. 

Enrollment  remains  open  for  the  gestational  diabetes  arm  until  the  end  of  July  2012. 


Subtask  2.2  Research  and  evaluate  the  impact  of  a  personal  health  record  (PHR)  on 

provider(s)  and  consumer(s)  with  particular  focus  on  chronic  disease  prevention. 

Core  online  PHR  functionality  includes: 

■  Request  Appointments 

■  Request  Prescription  Renewals 

■  Request  Lab  and  Test  Results 

■  Note  to  Doctor 

■  Note  to  Office 

■  Access  to  education  materials 

■  web  Visit® 
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Figure  1.  Relay  Health  PHR. 

It  is  important  to  note  that  some  participating  physicians  have  been  resistant  to  responding  to  subject 
messages  through  RelayHealth  as  reflected  by  qualitative  feedback  from  subjects.  This  finding  is 
not  surprising  as  physician  perception  of  PHR’s  are  likely  to  vary  from  total  abhorrence  to  complete 
support.  Physicians  have  stated  concerns  with  the  accuracy  and  privacy  of  PHRs  and  have 
expressed  doubts  that  patients  will  take  the  time  to  create  PHRs  and  keep  them  updated  (2). 

The  following  graph  shows  subject  usage  of  specific  PHR  message  types  (initiated  by  patient). 
Subjects  prefer  the  Rx  refill  requests,  request  lab/test  results,  and  “Note  to  Doctor”  functionality 
whereas  requesting  appointments  and  webVisits  are  used  less  frequently.  A  total  of  six  (6) 
subjects  have  not 
“linked”  to  their 
doctors  to  date  and 
are  unable  to  send 
messages.  All 
subjects  are 
provided  with 
written  instructions 
on  how  to  setup 
accounts  and  link  to 
their  doctor. 


PHR  Usage  by  Subjects 
June  2011  -  May  2012 


Figure  2.  Subject  PHR  Usage. 
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Subtask  2.3  Research  and  evaluate  the  impact  of  web-based  secure  messaging,  online 

consultations,  prescription  renewals,  and  appointment  scheduling  on  consumer 
awareness  and  their  ability  to  effectively  self  manage  their  health  compared  to 
those  consumers  not  using  a  PHR. 

Statistical  analysis  of  health  outcome  measures  for  the  PHR  Group  and  Control  Group  will  be 
completed  before  contract  termination.  Subject  outcome  measures  include  blood  pressure,  waist 
circumference,  weight,  body  mass  index  (BMI),  body  fat,  HDL  cholesterol,  triglycerides,  glucose 
and  hemoglobin  Ale. 

KEY  RESEARCH  ACCOMPLISHMENTS 


9  Completion  and  closure  of  study  A  -  15835.2 


Completion  and  closure  of  study  A  -  16192.1 


9  Enrolled  37  subjects  for  study  A  -15835.1 


REPORTABLE  OUTCOMES 


Conemaugh  presented  research  update  to  Dr.  Steve  Steffensen  and  Betty  Levine  during 
local  site  visit  in  Johnstown,  PA  on  June  9,  201 1 


9  Manuscript  submitted  to  Radiology  on  April  12,  2012  ;  not  accepted  for  publication 


9  8th  Annual  Conemaugh  Research  Poster  Symposium  (March  26-30,  2012) 


CONCLUSION 


The  EHR  study  and  PACS  study  have  both  been  completed.  Final  documents  were  submitted  to 
Conemaugh  and  U.S.  Army  Institutional  Review  Boards  (IRB)  and  found  to  be  acceptable.  A 
manuscript  detailing  the  PACS  study  was  submitted  to  Radiology  but  not  accepted  for 
publication. 

PHR’s  have  generated  minimal  interest  and  usage  by  subjects  and  providers  alike.  Our  findings 
align  with  patient  attitudes  nationwide.  Adoption  of  said  technology  remains  below  10%  and 
significant  usage  can  only  be  found  in  geographic  pockets  or  within  specific  patient  populations. 
As  the  nation  focuses  on  “meaningful  use”  of  EHR  systems,  wide-spread  adoption  of  PHRs  may 
be  5-10  years  away  from  reality  (3). 
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Appendix  1  -  Electronic  Health  Record  (EHR)  Implementation  Survey 


1.  The  instructions  and  prompts  are  helpful. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

12.0% 

3 

Sometimes 

52.0% 

13 

Occasionally 

4.0% 

1 

Most  of  the  time 

28.0% 

7 

Always 

4.0% 

1 

answered  question 

25 

skipped  question 

1 

2.  Getting  paper-based  documents  in  and  out  of  the  system  is  easy. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

3.8% 

1 

Sometimes 

19.2% 

5 

Occasionally 

19.2% 

5 

Most  of  the  time 

50.0% 

13 

Always 

7.7% 

2 

answered  question 

26 

skipped  question 

0 

3.  I  sometimes  wonder  if  I'm  using  the  right  command. 


Answer  Options 

Response 

Percent 

Response 

Count 

Never 

15.4% 

4 

Sometimes 

30.8% 

8 

Occasionally 

34.6% 

9 

Most  of  the  time 

19.2% 

5 

Always 

0.0% 

0 

answered  question 
skipped  question 

26 

0 

4.  The  speed  of  the  EHR  is  fast  enough  to  accomplish  tasks. 


Answer  Options 

Response 

Percent 

Response 

Count 

Agree 

42.3% 

11 

Disagree 

53.8% 

14 

Not  Sure 

3.8% 

1 

answered  question 

26 

skipped  question 

0 
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5.  This  software  seems  to  disrupt  the  way  1  normally  like  to  arrange  my  work. 

Answer  Options 

Response 

Percent 

Response 

Count 

Agree 

50.0% 

13 

Disagree 

38.5% 

10 

Not  Sure 

11.5% 

3 

answered  question 

26 

skipped  question 

0 

6.  The  organization  of  the  menus  or  information  lists  seems  quite  logical. 

Answer  Options  "g™  "SET 


Agree  50.0%  13 

Disagree  50.0%  13 

Not  Sure  0.0%  0 

answered  question  26 

skipped  question  0 


7.  It  is  relatively  easy  to  move  from  one  part  of  a  task  to  another. 

Answer  Options 

Response 

Percent 

Response 

Count 

Agree 

34.6% 

9 

Disagree 

57.7% 

15 

Not  Sure 

7.7% 

2 

answered  question 

26 

skipped  question 

0 

8.  It  is  easy  to  forget  how  to  do  things  with  the  EHR. 

Answer  Options 

Response 

Percent 

Response 

Count 

Agree 

34.6% 

9 

Disagree 

53.8% 

14 

Not  Sure 

11.5% 

3 

answered  question 

26 

skipped  question 

0 

9.  The  EHR  occasionally  performs  in  a  way  which  can't  be  logically  understood. 

Answer  Options 

Response 

Percent 

Response 

Count 

True 

69.2% 

18 

False 

30.8% 

8 

answered  question 

26 

skipped  question 

0 
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10.  I  have  to  seek  assistance  most  times  when  I  use  the  EHR. 


Answer  Options 

True 

False 


Response 

Percent 

16.0% 

84.0% 

answered  question 
skipped  question 


Response 

Count 


4 

21 


25 

1 


1 1 .  How  often  can  you  count  on  the  EHR  to  be  up  and  available. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

0.0% 

0 

Sometimes 

3.8% 

1 

Occasionally 

3.8% 

1 

Most  of  the  time 

73.1% 

19 

Always 

19.2% 

5 

answered  question 

26 

skipped  question 

0 

12.  How  often  is  the  EHR  subject  to  frequent  problems  and  crashes. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

7.7% 

2 

Sometimes 

23.1% 

6 

Occasionally 

69.2% 

18 

Most  of  the  time 

0.0% 

0 

Always 

0.0% 

0 

answered  question 

26 

skipped  question 

0 

13.  The  EHR  provides  me  with  all  the  information  1  need  to  take  care  of  the  patient. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

0.0% 

0 

Sometimes 

23.1% 

6 

Occasionally 

7.7% 

2 

Most  of  the  time 

61.5% 

16 

Always 

7.7% 

2 

answered  question 

26 

skipped  question 

0 
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14.  The  EHR  screens  include  a  lot  of  extra  information  that  1  don’t  need. 

Answer  Options 

Response 

Percent 

Response 

Count 

Agree 

19.2% 

5 

Disagree 

65.4% 

17 

Not  Sure 

15.4% 

4 

answered  question 

26 

skipped  question 

0 

15.  There  is  inaccurate  information  in  the  EHR. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

16.0% 

4 

Sometimes 

64.0% 

16 

Occasionally 

16.0% 

4 

Most  of  the  time 

4.0% 

1 

Always 

0.0% 

0 

answered  question 

25 

skipped  question 

1 

16.  The  EHR  provides  information  that  is  up-to-date. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

0.0% 

0 

Sometimes 

11.5% 

3 

Occasionally 

3.8% 

1 

Most  of  the  time 

76.9% 

20 

Always 

7.7% 

2 

answered  question 

26 

skipped  question 

0 

17.  The  system  lets  me  quickly  find  the  information  1  need. 

Answer  Options 

Response 

Percent 

Response 

Count 

Never 

0.0% 

0 

Sometimes 

40.0% 

10 

Occasionally 

28.0% 

7 

Most  of  the  time 

24.0% 

6 

Always 

8.0% 

2 

answered  question 

25 

skipped  question 

1 
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18.  The  information  in  the  EHR  is  presented  in  a  useful  format. 

Answer  Options 

Response 

Percent 

Response 

Count 

Yes 

30.8% 

8 

No 

15.4% 

4 

Depends  on  specific  functionality 

53.8% 

14 

answered  question 

26 

skipped  question 

0 

19.  The  information  in  the  system  includes  the  level  of  detail  that  1  need. 

Answer  Options 

Response 

Percent 

Response 

Count 

Yes 

42.3% 

11 

No 

15.4% 

4 

Depends  on  specific  functionality 

42.3% 

11 

answered  question 

26 

skipped  question 

0 
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20.  Compared  to  previous  routines,  how  has  the  EHR  changed  the  performance  of  the  following 

tasks? 

Answer  Options 

Much  More 

Slightly  More 

No 

Slightly 

Much 

N/A,  Don't 

Response 

Difficult 

Difficult 

Change 

Easier 

Easier 

Know 

Count 

Documenting  physical  exams 

3 

6 

1 

1 

4 

11 

26 

Documenting  histories 

4 

5 

1 

1 

5 

10 

26 

Documenting  allergies 

1 

2 

4 

2 

7 

9 

25 

Documenting  CPT  and  ICD-9  codes  for  billing  purposes 

2 

0 

3 

3 

4 

14 

26 

Keeping  problem  lists  updated 

2 

3 

2 

6 

6 

7 

26 

Keeping  medication  lists  updated 

3 

4 

3 

2 

6 

8 

26 

Ordering  laboratory  and  radiology  tests 

6 

2 

1 

2 

3 

12 

26 

Reviewing  laboratory  and  radiology  results 

2 

2 

3 

5 

7 

7 

26 

Writing  prescriptions 

2 

3 

1 

3 

7 

10 

26 

Renewing  prescriptions 

2 

1 

3 

3 

9 

8 

26 

Monitoring  medication  safety  during  prescribing 

1 

1 

1 

5 

4 

14 

26 

Monitoring  patient  medication  adherence 

1 

0 

4 

2 

3 

16 

26 

Communicating  referral  information  to  specialists 

2 

0 

4 

7 

2 

11 

26 

Reviewing  referral  information  from  specialists 

1 

1 

9 

5 

2 

8 

26 

Ordering  appropriate  preventive  care  services 

1 

4 

3 

3 

2 

13 

26 

Making  a  list  of  patients  based  on  diagnosis  or  history 

1 

0 

0 

3 

1 

21 

26 

Contacting  patients  to  remind  them  of  appointments 

0 

0 

4 

2 

2 

18 

26 

Assisting  patients  in  self-management  activities 

1 

1 

2 

2 

2 

18 

26 

answered  question 

26 
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21 .  How  strongly  do  you  agree  or  disagree  with  the  following  statements  regarding  the  EHR? 


Answer  Options 

Completely 

Generally 

Generally 

Completely 

Don’t 

N/A 

Response 

Agree 

Agree 

Disagree 

Disagree 

Know 

Count 

Using  the  EHR  has  enabled  me  to  accomplish  tasks 
quicker 

2 

11 

9 

4 

0 

0 

26 

1  work  longer  hours  to  see  the  same  number  of  patients 

8 

4 

4 

4 

2 

4 

26 

Using  the  EHR  has  enhanced  my  effectiveness  in  my  job 

2 

12 

8 

4 

0 

0 

26 

Using  the  EHR  has  made  it  easier  to  do  my  job 

3 

11 

8 

4 

0 

0 

26 

1  find  the  EHR  useful  in  my  job 

2 

16 

5 

2 

1 

0 

26 

Learning  to  operate  the  EHR  has  been  easy  for  me 

1 

15 

8 

2 

0 

0 

26 

1  have  become  skilled  at  using  the  advanced  features 

2 

12 

5 

0 

4 

3 

26 

Easier  to  access  patient  information  from  outside  the 
office 

2 

5 

1 

0 

3 

15 

26 

There  are  too  many  alerts  and  reminders 

3 

7 

3 

1 

4 

8 

26 

Has  decreased  the  amount  of  time  1  spend  talking  to 
patients 

4 

6 

7 

2 

3 

4 

26 

Helps  me  adhere  to  clinical  practice  guidelines 

1 

7 

2 

0 

6 

10 

26 

Using  an  EHR  has  caused  disruptions  to  my  work  flow 

4 

8 

8 

3 

1 

2 

26 

Has  improved  my  ability  to  make  decisions  about  patient 

1 

6 

4 

1 

3 

10 

25 

care 

Has  improved  my  ability  to  provide  preventive  care 

1 

6 

3 

2 

4 

10 

26 

1  can  better  monitor  how  many  of  my  patients  are  receiving 
appropriate  care 

1 

4 

5 

1 

2 

13 

26 

Benefits  of  adopting  an  EHR  have  outweighed  the 
challenges 

2 

10 

5 

3 

4 

2 

26 

answered  question 


26 
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22.  Would  you  recommend  EHRs  to  other  providers  interested  in  adopting  health 

information  technology? 

Answer  Options 

Response 

Percent 

Response 

Count 

Yes 

72.0% 

18 

No 

28.0% 

7 

answered  question 

25 

skipped  question 

1 

23.  If  you  could  change  one  thing  about  the  Allscripts  EHR  system, 

what  would  it  be? 

Answer  Options 

Response 

Count 

18 

answered  question 

18 

skipped  question 

8 

24.  What  has  been  the  most  positive  benefit  of  using  an  EHR? 

Answer  Options 

Response 

Count 

18 

answered  question 

18 

skipped  question 

8 

25.  How  often  do  you  print  out  a  Visit  Summary  report  from  the  EHR  system  for 
your  patient  at  the  conclusion  of  the  patient  visit? 


Answer  Options 

Response 

Percent 

Response 

Count 

Always 

0.0% 

0 

Sometimes 

20.0% 

5 

Never 

44.0% 

11 

Don’t  know 

12.0% 

3 

Not  Applicable 

24.0% 

6 

answered  question 

25 

skipped  question 

1 

26.  How  long  have  you  been  using  the  Allscripts  EHR  at  your 

practice? 

Answer  Options 

Response 

Count 

26 

answered  question 

26 

skipped  question 

0 
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27.  How  would  you  classify  your  level  of  comfort  with  general  computer  technology 

(e.g.  email,  Internet,  word  processing)? 

Answer  Options 

Response 

Percent 

Response 

Count 

Very  comfortable 

53.8% 

14 

Somewhat  comfortable 

46.2% 

12 

Not  very  comfortable 

0.0% 

0 

answered  question 

26 

skipped  question 

0 

28.  Please  choose  the  statement  that  best  describes  the  training  you  received  for 

your  current  EHR. 

Answer  Options 

Response 

Percent 

Response 

Count 

1  have  received  no  training  and  I’m  learning  the 
system  as  1  use  it 

0.0% 

0 

Informal  training  by  practice  staff  when  time 
permitted 

3.8% 

1 

Less  than  ten  hours  dedicated  to  formal  training 
(with  vendor  or  practice  trainers) 

53.8% 

14 

Ten  or  more  hours  dedicated  to  formal  training  (with  „0/ 

11 

vendor  or  practice  trainers) 

answered  question 

26 

skipped  question 

0 

29.  What  is  your  title? 

Answer  Options 

Response 

Percent 

Response 

Count 

Clerical 

37.5% 

9 

Nurse 

33.3% 

8 

Office  Manager 

4.2% 

1 

Physician 

8.3% 

2 

Physician  Assistant 

0.0% 

0 

Other 

16.7% 

4 

answered  question 

24 

skipped  question 

2 

30.  Do  you  have  any  previous  EHR  experience? 


Answer  Options 

Yes 

No 


Response 

Percent 

26.9% 

73.1% 

answered  question 
skipped  question 


Response 

Count 


7 

19 


26 

0 
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31 .  How  satisfied  are  you  with  the  EHR  system? 


Answer  Options 

Response 

Percent 

Response 

Count 

Very  Dissatisfied 

11.5% 

3 

Dissatisfied 

23.1% 

6 

Neutral 

30.8% 

8 

Somewhat  Satisfied 

30.8% 

8 

Very  Satisfied 

3.8% 

1 

answered  question 

26 

skipped  question 

0 

32.  Please  provide  additional  comments  as  needed. 

Answer  Options 

Response 

Count 

answered  question 

6 

6 

skipped  question 

20 
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Appendix  2  -  Protocol  Closure  Memorandum 


Pa^e  I  of  I 


John  Hargreaves 


RWTl  i  SnoBch,  La ua  R  Dr  CIV  USA  MEDCOM  USAMRMC  [Laufa.Brasch@us.army. mil] 

Sent!  Thursday ,  April  IB,  2012  2:16  PM 
TO:  Brian  Liab 

CC:  Richard  WazNak:  Bennetf,  Jotf  H  Ms  CIV  USA  MEDCOM  USAMRMC;  -Stephenson,  Jeffray  Dr  IBA';  tra  tty.  lari  no#  teEte.org'; 

Bane.  Elena  G  Ms  CIV  USA  ME  DCOM  USAMRAA;  John  Hargreaves:  Duchasneau,  Caryn  L  Ms  CIV  USA  MEDCOM 
U  SAM  BMC;  Katopol,  Kristen  R  Ms  CTR  US  USA  MEDCOM  USAMRMC:  Englai,  NlarKy  E  CTR  US  USA  MEDCOM 
USAMHMC;  Drayton,  Maria  Ms  CTR  US  USA  MEDCOM  USAMRMC;  Brosch,  Laura  R  Dr  CIV  USA  MEDCOM  USAMRMC 

Sutflect:  A- 15836.2,  Pfetocol  CbBure  Memorandum  (Propose  Log  Number  09064002,  Award  Number  WB 1  XWH-OB-2-OOE1 ) 
(UNCLASSIFIED) 

ClassificadoD:  IKCLASSIFIED 
Caveats:  NONE 

SUBJECT:  Project  Completion  for  the  Protocol,  "Military  Interoperable  Digital  Hospital 
Testbed  (MIDHT}  Year  2  Arm  1 :  Longitudinal  Study  for  the  Use  of  Ambulatory  Electronic 
Health  Records  (EH Ft)  in  Rural  Communities,"  Submitted  by  Brian  Lieb,  DO, 

Conemaugh  Valley  Memorial  Hospital,  Camolltown,  Pennsy  lvania,  in  Support  of  the 
Proposal,  "Military  Interoperable  Digital  Hospital  Testbed  (MIDHT),  Submitted  by 
Richard  S.  Wozniak.  MD,  Memorial  Medical  Center,  Johnstown,  Pennsylvania,  Proposal 
Log  Number  09064002,  Award  Number  Wei XWH-09- 2- 0061,  HRPO  Log  Number  A- 
1 5835.2 


1 .  A  final  report  was  received  by  the  U.S.  Army  Medical  Research  and  Materiel 
Command  (USAMRMC),  Office  off  Research  Protections  (GRP),  Human  Research 
Protection  Office  (HRPO)  on  1 2  April  2012.  This  no  greater  than  minimal  risk  study  was 
initially  approved  by  the  HRPO  on  10  May  2010. 

2.  The  Conemaugh  Memorial  Medical  Center  Institutional  Review  Board  documentation 
acknowledging  closure  of  this  protocol,  dated  2  April  2012,  was  received  by  the 
USAMRMC  GRP  HRPO  on  12  April  2012.  The  final  report  and  supporting  documents 
were  reviewed  and  found  to  be  acceptable. 

3.  No  further  review  of  the  protocol  will  be  conducted,  and  the  HRPO  protocol  file  will 
be  closed. 

4.  The  HRPO  point  of  contact  for  this  study  is  Nancy  Enalar,  MHL,  BSN,  RN,  Human 
Subjects  Protection  Scientist,  301-61 9-224^ r a e.eiiala^.ctr@us.armv.miL 


LAURA  R.  BROSCH,  PhD 
Director,  Office  of  Research  Protections 
Director,  Human  Research  Protection  Office 
U.S.  Army  Medical  Research  and  Materiel  Command 

Note:  The  official  copy  of  this  closure  memo  is  housed  with  the  protocol  file  at  the 
Office  of  Research  Protections,  Human  Research  Protection  Office,  504  Scott  Street, 
Fort  Detrick,  MD  21702.  Signed  copies  will  be  provided  upon  request. 

Claaific-atioD:  UNCLASSIFIED 
Caveats:  NONE 


6/19/2012 
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Appendix  3  -  Protocol  Closure  Memorandum 
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John  Hargreaves 


From:  Duchesirasu,  Csiyn  L  Me  CIV  USA  ME  DCOM  USAMRMC  [Caiyn.DLK4*esneaLJ#u&.BnTTy.miI] 

Sent:  Monday,  Aiqusl  29,  201 1  6:03  PM 

TO:  Richard  Wozniak 

CC:  John  Hargreaves:  Bennett,  Jod  H  Ms  CIV  USA  MEDCOM  USAMRMC;  John  Karducfe;  'Stephenson,  Jeffrey  Dr  IEA';  Jeanette 

Croneir;  CliriB  Smith:  Thomas  Simunich;  Wendi  Nagle:  Bane,  Elena  G  Ms  CIV  USA  MEDCOM  U5AMFAA: 

'st&'e.stefle  rse  r@tatTC.Drg':  Broach,  Laura  R  Or  CIV  USA,  MEDCOM  USAMRMC;  Duchesneau,  Caiyn  L  Me  CIV  USA 
MEDCOM  USAMRMC;  Kalopd,  Kristen  R  Ma  GIR  US  USA  MEDCOM  USAMRMC:  Eaton,  Karen  M  Ms  CTR  US  USA 
MEDCOM  USAMRMC;  DrHyton,  Maria  Ma  CTR  US  USA  MEDCOM  USAMRMC;  Dyaon,  Nioole  CIV  US  USA  MEDCOM 
USAMRMC 

SLtttKt  A-1G192.1,  Prolaool  Closure  Memorandum  {Proposal  Log  Number  10322003,  Award  Number  WB1XWH-10-2-Q1BD) 
^UNCLASSIFIED) 


Oassiffcatiori:  UNCLASSIFIED 

Carveais:  NONE 

SUBJECT:  Project  Completion  for  the  Protocol,  “Military  Interoperable  Oigftal  Hospital  Testbed  (MIDHT)  Year  2,  Arm 
1 :  Sy  stem- Wide  Imago  Access:  Analysis  o,n  Duplicate  Testing  in  a  Rural  Healthcare  Environment,"  Submitted  ty 
Richard  S.  Woznrak,  ndD,  Memorial  Medical  Center,  Johnstown,  Pennsylvania,  in  Support  of  the  Proposal,  “Military 
Interoperable  Digital  Hospital  Testbed  (MIDHT),"  Submitted  by  John  Karduck,  MD,  Memorial  Medical  Center, 
Johnstown,  Pennsylvania,  Proposal  Log  Number  10322003,  Award  Number  W81XWH- 10-2-01  BO,  HRPO  Log 
Number  A- 161 92.1 


1 .  A  final  report  and  request  to  dose  the  protocol  was  received  by  the  U.S.  Army  Medical  Research  and  Materiel 
Command  (USAMRMC),  Office  of  Research  Protections,  Human  Research  Protection  Office  (HRPO)  on  27  July 
2011.  This  no  greater  than  minimal  risk  study  was  initially  approved  by  the  HRPO  on  IS  August  201 0. 

2.  The  Memorial  Medical  Center  Institutional  Rariew  Board  documentation  acknowledging  dosure  of  this  protocol, 
dated  14  July  201 1 ,  was  received  by  the  USAMRMC  HRPO  on  27  July  201 1 .  The  final  report  and  supporting 
documents  were  reviewed  and  found  to  be  acceptable. 

3.  No  further  review  of  the  protocol  will  be  conducted,  and  the  HRPO  protocol  tile  will  be  dosed. 

4.  The  HRPO  point  of  contact  for  this  study  is  Karen  M.  Eaton,  MS,  Human  Subjects  Protection  Scientist  at  301- 
61 9-  9266/karen.  m.eaton@  us  .army  .m  il  „ 


CARYN  L.  DUCHESNEAU,  CIP 

Chief,  Human  Subjects  Protection  Review 

Human  Research  Protection  Office 

Office  of  Research  Protections 

U.S.  Army  Medical  Research  and  Materiel  Command 

Note:  The  official  copy  of  this  dosure  memo  is  housed  with  the  protocol  file  at  the  Office  of  Research  Protections, 
Human  Research  Protection  Office,  504  Scott  Street,  Fort  Detrick,  MD  21702.  Signed  copies  will  be  provided  upon 
request. 

Gasufkation:  UNCLASSIFIED 
Caveats:  NONE 


S/3  Of  20 1 1 
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Full  Title  Page 

Richard  S.  Wozniak,  MD,  John  S.  Hargreaves,  MBA,  Thomas  J.  Simunich,  MBA,  MS  in  MIS, 
Lisa  Pasierb,  Ph.D,  Jeanette  R.  Croner,  MHA. 

Conemaugh  Memorial  Medical  Center 
1086  Franklin  Street 
Johnstown,  Pennsylvania  15905 

RW  -  rwozniak@conemaugh.org,  814-534-9106 
JH  -  jhargrea@conemaugh.org,  814-269-5277 
TS  -  tsimunic@conemaugh.org,  814-269-5246 
LP  -  lpasierb@conemaugh.org,  814-269-5248 
JC  -  icroner@conemaugh.org,  814-269-5247 


The  Impact  of  System-Wide  Image  Access  on  Duplicate  Testing  in  a  Rural  Healthcare  System. 

This  paper  has  not  been  presented  at  an  RSNA  meeting  and  has  not  been  accepted  for 
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Abbreviated  Title  Page 


a)  The  Impact  of  System-Wide  Image  Access  on  Duplicate  Testing  in  a  Rural  Healthcare 
System. 

b)  Original  Research 

c)  Advance  in  Knowledge 

1 .  The  ability  for  the  receiving  institution  to  view  images  taken  off-site  through  a 
picture  archiving  and  communications  system  (PACS)  resulted  in  a  7%  reduction 
in  duplicate  chest  x-ray  and  CT  scan  (head)  testing  from  0-7  days  when  compared 
to  a  pre  PACS  implementation  period. 

2.  As  surveyed,  seventy  percent  of  physicians  positively  stated  that  immediate  image 
access  did  reduce  the  number  of  exams  reordered. 

d)  Implications  for  Patient  Care 

1 .  System- wide  availability  of  images  will  have  a  positive  impact  and  may  reduce 
unnecessary  imaging  at  the  receiving  institution.  Benefits  will  also  transfer  to 
patients  as  health  risks  associated  with  radiation  exposure  will  decrease. 

2.  Implemented  should  have  a  solid  plan  in  place  to  communicate  and  train  users  on 
the  new  functionality.  Leadership  should  expect  that  change  management  will 
vary  amongst  physicians  and  may  impact  immediate  results. 

e)  Summary  Statement 

The  implementation  of  picture  archiving  and  communications  system  (PACS)  throughout  a 
rural  health  care  system  will  have  positive  benefits  both  to  patients  and  the  institutions 
involved  and  likely  reduce  unnecessary  imaging,  operating  costs  and  radiation  exposure  to 
patients. 
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Abstract 


Purpose: 

To  test  the  hypothesis  that  duplicate  imaging  will  decrease  among  transfer  patients  to  a  tertiary 
care  center  from  two  rural  off-site  hospitals  after  the  implementation  of  picture  archive  and 
communications  system  (PACS)  at  the  off-sites. 

Materials  and  Methods: 

This  minimal  risk,  HIPAA-compliant  study  was  approved  by  Conemaugh’s  Institutional  Review 
Board  (IRB),  with  waiver  for  informed  consent  for  retrospective  review  of  medical  records. 

Using  a  master  patient  index  (MPI),  625  duplicate  chest  x-rays  and  CT  scans  of  the  head  between 
sending  and  receiving  institution  (taken  within  0-7  days)  were  collected  from  July  2008  to  June 
2010.  The  study  design  utilized  a  pre  vs.  post  quantitative  methodology,  with  time  periods  based 
upon  the  extension  of  PACS  technology  to  off-site  locations.  Additionally,  qualitative  feedback 
was  gathered  from  physicians  (n=76)  using  a  survey  tool  to  assess  the  impact  of  immediate  image 
access  and  to  quantify  the  number  of  off-site  studies  viewed  by  physicians  (n=70)  with  PACS 
access. 

Results: 

A  Chi-Square  test  of  independence  applied  to  either  Days  between  date  of  service  or  aggregated 
Days  between  date  of  service  over  time  period  did  not  yield  a  statistically  significant  result 
despite  a  7%  reduction  in  duplicate  imaging  (hypothesis  not  accepted).  A  financial  analysis  of 
the  resulting  seven  percent  reduction  in  duplicate  tests  suggests  a  savings  of  $187,075  to  patients 
and/or  insurance  companies. 

Conclusion: 

Extension  of  PACS  technology  to  referring  institutions  is  beneficial;  however,  realization  of  a 
significant  reduction  in  duplicate  testing  will  depend  upon  full  support  of  ordering  physicians, 
proper  training,  and  effective  communication. 
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Introduction 


The  United  States  spends  more  on  health  care  than  any  other  country  with  an  annual  average  of 
$6,401  per  person,  which  is  2.4  times  the  average  of  developed  countries  (1).  As  hospital 
reimbursement  becomes  more  challenging,  health  information  technology  (HIT)  may  offer 
solutions  to  achieve  organization-wide  cost  savings.  Duplicate  testing  is  not  only  a  well-known 
source  of  extraneous  health  care  expenses  but  may  also  pose  additional  radiation  risks  to  patients. 
As  a  rural  health  care  system  with  multiple  referring  hospitals,  how  can  efficiency  and  patient 
safety  improve  through  coordinated  care? 

Duplicate  testing  remains  an  industry-wide  challenge  that  must  be  addressed  in  order  for  health 
care  reform  to  be  realized.  Haley  et  al  found  that  53%  of  transferred  trauma  patients  had  some 
portion  of  their  images  duplicated;  resulting  in  $650,000  in  additional  costs  (2).  Thomas  et  al 
presented  similar  findings  in  that  43%  of  patients  had  computed  topography  (CT)  scans  repeated 
during  facility  transfers  (3). 

Recent  studies  have  stressed  the  inherent  risk  to  patients  due  to  increased  source  of  radiation 
exposure.  Brenner  and  Hall  state  that  62  million  CT  scans  are  performed  annually  in  the  United 
States  and  involve  larger  radiation  doses  than  more  conventional  x-ray  tests  (4).  Berrington  de 
Gonzalez  et  al  also  provide  support  for  increased  cancer  risk  estimating  that  29,000  future 
cancers  could  be  related  to  CT  scans  (5).  Sodickson  et  al  recommend  the  quantity  of  imaging 
should  be  monitored  over  time  to  ensure  minimization  of  radiation  exposure  (6). 
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One  solution  that  may  reduce  unnecessary  radiology  testing  and  reduce  radiation  exposure 
between  central  and  remote  sites  is  called  Picture  Archiving  and  Communication  System 
(PACS).  PACS  are  computer  networks  dedicated  to  the  storage,  retrieval,  and  presentation  of 
images  produced  by  medical  imaging  devices.  PACS  replaces  film  archives,  allowing  imaging 
access  simultaneously  and  from  off-site  locations.  PACS  is  commonly  believed  to  reduce  the 
number  of  unnecessary  duplicate  imaging  tests  ordered  because  of  originals  being  lost  or  stored 
at  a  remote  location.  Past  research  suggests  an  individualized  evaluation  of  PACS  technology 
where  incidence  of  duplicate  testing  may  be  high  (7).  Institutions  involved  with  the  transfer  of 
patients  may  have  the  most  to  gain  through  this  process. 

Materials  &  Methods 

This  study,  located  within  the  Conemaugh  Health  System  (CHS),  analyzed  the  impact  of 
extending  PACS  to  two  remote  hospitals  stretched  over  a  two  county  area  in  southwestern 
Pennsylvania.  Conemaugh  Memorial  Medical  Center  (central  site),  located  in  Johnstown, 
Pennsylvania,  is  a  tertiary  care  regional  referral  hospital  known  for  clinical  excellence  and  patient 
satisfaction.  The  level  1  trauma  center  located  at  the  central  site  is  one  of  just  eleven  centers  in 
Pennsylvania.  Miners  Medical  Center  (remote  site  1)  is  a  30-bed  community  satellite  hospital 
located  45  minutes  to  the  north  of  Johnstown  whereas  Meyersdale  Medical  Center  (remote  site  2) 
is  a  20-bed  Critical  Care  Access  hospital  located  60  minutes  to  the  south  of  Johnstown.  The 
study  was  approved  by  Conemaugh’ s  Institutional  Review  Board  (IRB),  which  included  a  waiver 
of  informed  consent  for  retrospective  review  of  medical  records  and  a  minimal  risk  designation. 
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Implementation 

Conemaugh  extended  the  McKesson  PACS,  Radiology  Information  System  (RIS)  and  Dolby 
Digital  Dictation  system  used  at  the  central  site  to  the  two  remote  facilities  (Figure  1).  This 
project  allowed  CHS  to  achieve  consistency  of  radiology  imaging,  report  management,  and 
image  access  across  the  health  system.  PACS  went  live  at  the  remote  sites  on  July  1,  2009  (image 
transfer  only).  During  the  next  six  months,  activities  were  completed  to  seamlessly  integrate 
both  the  RIS  and  PACS  systems  for  sharing  of  reports  in  production  use. 


Study  Design 

Both  qualitative  and  quantitative  methods  were  employed  to  investigate  the  study  objectives  and 
the  hypothesis.  Researchers  hypothesized  that  the  number  of  duplicated  chest  X-rays  and  CT 
scans  (head)  would  significantly  decrease  after  implementation  of  each  phase  of  the  PACS 
implementation  compared  to  that  of  baseline. 

Data  collection  primarily  utilized  historical  data  (retrospectively)  to  obtain  the  number  of 
duplicated  diagnostic  imaging  tests  by  evaluating  empirical  data  retrieved  from  hospital  financial 
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systems.  The  data  was  limited  to  patients  that  were  first  treated  at  one  of  the  two  remote  sites  and 
subsequently  treated  at  the  central  site  within  the  defined  duplicate  test  time  frame  (0-7  days). 
This  comparative  analysis  used  a  PRE  (before  PACS)  and  POST  (after  PACS)  time  period,  see 
Table  I,  design  to  ascertain  the  expected  change  in  the  viewing  of  PACS  images.  The  time 
periods  were  chosen  with  consideration  for  symmetry  of  data  sets,  stabilization  of 
implementation,  and  the  possibility  of  seasonality  (time)  as  a  covariate. 


PRE  (baseline) 

POST 

Phase  1  i  Phase  2 

2008 

2009  |  2010 

July  through 
December 

January  through 
June 

July  through  j  January  through 

December.  ,  June 

Table  I.  Study  Timeline. 

Conemaugh  MIS  department  provided  duplicate  test  reports  to  the  study  team  for  analysis  using 
the  Master  Patient  Index  (MPI)  software.  The  report  included  chest  x-rays  (CPT  71010  and 
71020)  and  CT  scans  of  the  head  (CPT  70450)  for  the  stated  time  periods.  The  report  included 
data  for  patients  that  had  the  same  test,  as  specified  by  the  CPT  codes  above,  and  determined  by 
the  patient  identifier,  date  of  service,  and  service  location.  Inclusion  of  data  was  limited  to 
studies  repeated  at  the  central  site  within  0-7  days.  Time  of  service  was  used  to  include/exclude 
tests  performed  on  the  same  date.  Additional  images  completed  at  CMMC  were  removed  from 
the  analysis.  Summary  radiology  volume  data  was  also  collected  to  assess  the  level  of 
consistency  of  volume  over  time  and  thereby  providing  one  measure  of  the  homogeneity  of  the 
pre  and  post  data  sets. 
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In  order  to  understand  actual  MMC  physician  usage  of  studies  performed  at  the  two  rural 
hospitals,  PACS  User  Reports  were  provided  to  the  study  team  for  analysis.  The  population 
included  a  randomized,  proportionate  sample  of  active  physicians  on  the  MMC  medical  staff. 
Reports  were  prepared  for  October  -  December  2009  and  April  -  June  2010,  which  detailed 
viewing  of  studies  that  originated  from  the  remote  sites. 

A  physician  opinion  survey  was  also  distributed  to  all  applicable  physicians  on  the  MMC 
medical  staff  in  traditional  hard  copy  and  online  form.  The  survey  was  designed  by  Canada 
Health  Infoway  and  was  modified  for  use  by  Conemaugh  with  permission.6  The  survey  was 
designed  to  gather  qualitative  feedback  from  physicians  that  use  the  system  daily  in  their  course 
of  patient  care.  Survey  objectives  included  assessing  the  impact  of  immediate  image  access  to 
studies  performed  at  two  rural  hospitals  on  productivity,  decision-making,  patient  transfers,  and 
duplicated  exams. 

Results 

A  Chi-Square  test  of  independence  applied  to  either  Days  between  (b/w)  date  of  service  (DOS) 
or  aggregated  Days  b/w  DOS  over  time  period  does  not  yield  a  statistically  significant  result 
despite  a  7%  reduction  in  duplicate  imaging  (hypothesis  not  accepted).  However,  a  Chi-Square 
test  of  goodness-of-fit  for  aggregated  Days  b/w  DOS  using  pre  data  as  the  expected  (population) 
values  and  the  post  data  as  observed  values  does  yield  a  statistically  significant  result  at  a  family- 
wise  alpha  =  0.05.  This  result  implies  that  the  implementation  of  the  PACS  system  at  the  remote 
sites  did  contribute  to  the  change  in  the  distribution  of  the  count  of  duplicative  testing  (Figure  2). 
Furthermore,  no  statistical  significance  was  found  by  CPT  code  or  location  (Figures  3-4). 
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Radiology  volume  data  by  CPT  code  for  the  remote  sites  is  homogeneous  between  the  pre  and 


post  data  sets. 


220- 


Days  b/w  DOS,  aggregated 


Figure  2.  Duplicated  Tests  (PRE  vs.  POST)  by  Days  Between  Date  of  Service. 
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Figure  3.  Duplicated  Tests  (PRE  vs.  POST)  by  CPT  Code. 
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Figure  4.  Duplicated  Tests  (PRE  vs.  POST)  by  Initial  Location. 
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Despite  non-significant  findings  in  the  reduction  of  duplicate  testing,  the  financial  analysis  of  the 
resulting  seven  percent  reduction  in  duplicated  chest  x-rays  and  CT  scans  of  the  head  suggests  a 
savings  of  $187,075  to  patients  and/or  insurance  companies.  As  healthcare  costs  continue  to 
climb  and  funding  becomes  more  restricted,  hospitals  and  health  systems  must  consider 
implementing  health  information  technologies  to  improve  efficiencies  and  save  money.  Though 
not  specifically  addressed  in  this  study,  the  reduction  in  duplicate  testing  will  translate  into  less 
radiation  exposure  opportunities  for  patients  transferred  between  facilities. 

The  following  data  was  collected  for  a  randomized,  proportionate  sample  of  70  physicians  with 
PACS  access.  As  depicted  below  in  Figure  5,  the  most  active  users  of  the  PACS  system  in  terms 
of  viewing  studies  originating  from  MIMC  and  MYMC  are  Emergency  Medicine  (n=65)  and 
Trauma  (n=51)  physicians.  This  result  is  expected  as  MMC  is  a  tertiary  care  referral  hospital  with 
a  Level  1  trauma  center.  The  next  grouping  includes  Otolaryngology  (n=35).  Urology  (n=34), 
and  Pulmonary  (n=31).  A  third  grouping  includes  General  Surgery  (n=28),  Orthopedics  (n=27) 
and  Neurosurgery  (n=26).  The  graph  does  not  include  an  outlier  for  an  orthopedic  surgeon 
(n=581). 
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Studies  from  Remote  Sites  Viewed  via  PACS 
Random  Proportionate  Sample  of  Medical  Staff 
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Figure  5.  Number  of  studies  performed  off-site  viewed  using  PACS. 

Qualitative  feedback  from  system  users  was  collected  via  a  survey  tool.  Seventy  six  physicians 
completed  the  survey  during  October  2010  through  January  2011,  representing  approximately  a 
37%  response  rate.  The  following  graph  depicts  self  reported  frequency  of  electronic  access 
(Frequently,  Sometimes,  Seldom,  or  Never)  of  images  originating  from  MIMC  and  MYMC  by 
stated  specialty.  Results  do  align  with  empirical  usage  data  for  highest  volume  specialties: 
General  Surgery,  Trauma,  Emergency  Medicine,  Internal  Medicine,  Orthopedics,  Urology  (not 
shown),  Neurosurgery,  Otolaryngology  (not  shown),  and  Pulmonary  (not  shown).  Family 
practitioners  responded  the  most  but  actual  usage  of  the  system  is  limited. 
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Figure  6.  Survey  Respondents  by  Specialty  and  Frequency  of  PACS  Usage. 


Surveys  that  indicated  that  the  physician  never  used  PACS  to  access  images  from  the  remote 
sites  were  removed  from  the  dataset  before  analysis.  The  remaining  responses  (n=55)  formed  the 
dataset  of  analysis. 


The  most  important  survey  question  in  terms  of  the  hypothesis  was  Question  10a:  Immediate 
PACS  image  access  to  films  performed  at  Miners/Meyersdale  has  reduced  the  number  of  exams 
reordered  because  the  images  were  not  available  when  I  needed  them?  Seventy  percent  (70%)  of 
physicians  that  use  the  PACS  system  for  this  purpose  either  Strongly  Agreed  or  Moderately 
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Agreed  that  the  number  of  duplicated  tests  has  been  reduced  post  implementation  whereas  the 


remaining  30%  disagreed  or  stated  it  was  not  applicable. 


Physician  Opinion 

Figure  7.  Self-reported  Physician  Opinion  on  Duplicate  Testing  by  Frequency  of  PACS  Usage. 


Hypothesis  testing  of  this  question  and  others  indicate  that  the  response  per  category  most 
probably  represents  a  real  difference  in  respondent  opinion.  The  results  of  the  hypothesis  testing 
do  not  change  for  the  aggregations  of  the  responses  of  those  questions.  [Asymptotic  significance 
at  the  0.05  level  is  shown  for  all  hypothesis  testing.]  Questions  were  analyzed  using  the  original 
scale,  a  5  level  Likert  scale,  and  an  aggregation  (collapsing)  of  that  scale  comprised  of  3  levels 
(Agree,  Disagree,  N/A). 
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Discussion 


The  implementation  of  PACS  technology  at  smaller  hospitals  within  the  same  health  system  is 
beneficial  and  likely  to  reduce  costs  and  radiation  exposure  to  patients.  This  study  found  that 
positive  qualitative  feedback  and  modest  system  usage  by  physicians  did  translate  into  a 
reduction  in  duplicate  testing  although  not  significant  for  patients  receiving  care  from  multiple 
facilities  within  the  same  rural  health  system.  Despite  new  technologies  being  available, 
physicians  must  take  the  appropriate  amount  of  time  to  receive  education  and  attend  training 
sessions.  Unfortunately,  some  physicians  may  not  change  their  ordering  habits,  which  will 
reduce  the  expected  benefit  significantly  depending  on  position  and  corresponding  volume. 

One  limitation  of  the  study  is  that  the  reason  for  ordering  the  test  was  not  consistently  collected 
and  therefore  unavailable  for  analysis  by  investigators.  Such  analysis  would  have  been  beneficial 
to  determine  how  many  more  duplicate  tests  could  have  been  prevented  if  the  physician  had  been 
aware  of  the  previous  test  conducted  off-site  or  the  test  was  re-ordered  based  upon  sound  clinical 
judgment. 

To  enhance  data  analysis  and  study  conclusions,  future  research  should  attempt  to  collect 
reasoning  information  used  in  deciding  to  reorder  an  image.  Given  the  surge  in  new  technology 
investments  nationwide  from  American  Recovery  and  Reinvestment  Act  (ARRA)  funding,  future 
studies  should  be  conducted  on  health  information  exchanges  (HIE)  that  share  images  across  the 
street  or  across  state  lines.  An  understanding  of  the  cost  benefit  analysis  of  said  HIE  projects  will 
be  important  to  move  the  industry  forward. 
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